We consider a gauge symmetry in a quantum Hilbert space. The symmetry leads to that of the heat-kernel and of the anomaly formulae which were previously obtained by the authors. This greatly simplifies and clarifies the structure of the formulae. We explicitly obtain the anomaly formulae in two and four dimensions, which ``unify'' all kinds of anomaly. The symmetry corresponds to that of the counterterm formulae in the background field method. As an example, the non-abelian anomaly is considered.